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Abstract 

A result is given to find points where a real valued function on the plane is not smooth. 
Provided this function is induced by a smooth mapping from three dimensions to the 
plane, from a function on surfaces in three dimensions. This has applications to numerical 
methods such as image processing. 
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1. introduction 

A function / : R m — > K is continuous at x £ R m if for every sequence Xi converging 
to x, the sequence f(xi) converges to f(x). Thus any subsequence of any sequence /(a;,*) 
converges to the same limit, namely f(x). 

Unfortunately, the definition of continuity is not helpful as to how we may locate a 
discontinuity in an image, which is a finite array of measurements. This paper presents a 
mathematical result which enables one to find points where a function I : R 2 — > R is not 
smooth, such as discontinuities. It is assumed that I is induced by a smooth mapping 

t:R 3 -^R 2 dx,y,z) = {-,-) z^Q 

z z 

In previous work a problem which arose in computer vision was formulated using 
the shape spaces introduced by Kendall. An application of the explicit solution was the 
following. An image is acquired from a video camera. The image contains a shape such as 
a triangle, square or circle. The problem is to classify it into categories such as triangles, 
squares or circles. For example, an image of a triangle will typically contain far more 
than three points, so it needs to be simplified. A square on the other hand should not be 
simplified to a triangle, and a circle should not be simplified very much at all. In order to 
obtain the points to be simplified in the first place, we first have to find discontinuities 
in /. 

This paper is concerned with the mathematical aspects of the discontinuity identi- 
fication problem. In what follows it is assumed the reader is familiar with differential 
geometry. Although the result is motivated by computer vision and human vision, dif- 
ferential geometry is not familiar there. As a mathematical result it could be extended 
by a wider scientific audience by replacing the above t with some other measurement t. 

It is very surprising that using optical principles, we can establish mathematically that 
there really is a discontinuity. Not some other property such as a local maximum in a 
derivative, but a discontinuity. The derivative is denned in calculus or real analysis. All 
books on real analysis [2] make it clear that continuity is a necessary condition for the 
derivative to be well-defined. 

When grappling with this problem others have used piecewise smooth or other as- 
sumptions. Instead here the contrapositive will be used to obtain a more general result. 
Owing to the microstructure of a scene surface, the light radiated from the surface un- 
dergoes extremely rapid intensity changes over small regions. This can be confirmed by 
examining gray values of neighbouring points in a real image. It is for this reason that 
this more general result may be helpful. 

A fundamental psychological observation about the nature of vision was made by 
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Wertheimer, who noticed the apparent motion not of individual dots, but of fields in im- 
ages presented sequentially as a movie [16j. This started the Gestalt school of psychology 

CEZlCEo]. 

A differential operator maps a greyscale image to a vector field. It is implicitly suggested 
by Hoffman [8 J that the primary visual cortex of the brain produces some sort of vector 
field. It should be possible to line up the vectors in a head to tail fashion to form contours. 
We want to know when this results in a family of closed oriented contours rather like a 
topographic map. The answer is provided by a famous theorem of differential geometry, 
namely the Foliation theorem of Frobenius. 

2. Sufficient condition for discontinuity 

In what follows we consider a real valued function on the plane. Some examples of such 
functions in a color image are red, blue, green, hue, saturation or brightness. 

example 1. Suppose / is a smooth function (intensity). Let M be the subset of the 
plane on which (—dl/dy,dl/dx) is non-zero. M is an open set because by definition of 
continuity the inverse of a continuous function maps open sets to open sets, and the line 
minus zero is open. The map 

(x, y) \-> span(- |^ (x, y), y)) 

is a smooth function that maps M to the set of lines through the origin in the plane P 1 
(projective space). 

It is an example of what is called a one dimensional distribution in differential geometry 
[14|, 2j The vector field is called a basis for this distribution. 

example 2. Consider f(x, y) = x + y and g(x, y) = e x+v . 

dy dy dx dx 

Thus at every point in the plane, the Hamiltonian vector of g is e x+v times the Hamil- 
tonian vector of /. These are two different vector fields that are a basis for the same 
distribution. On the other hand the Hamiltonian vector of e x + e y is not collinear with 
the Hamiltonian of /, so not every pair of vector fields does span a one dimensional 
distribution. 

definition 1. Suppose Y is a vector field, and it is the basis for a distribution. The 
distribution is called integrable if at every point one can find a disk on which a coordinate 
system (xi,X2) can be chosen such that 

Y 9 +Y 9 - 9 
*i ~ r X2T, — — q — 

oyi oy 2 ox! 
The following is based on proposition 1.53 in [14j . 

LEMMA 1. Let m £ M, and let X be a smooth vector field on M such that X(m) ^ 
0. Then there exists a coordinate system (U,ip) with coordinate functions x\, x-i on a 
neighborhood of m such that 



(2-1) 
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Proof. Choose a coordinate system (V, r) centered at m with coordinate functions 
2/1, j/2 , such that 



dyi 



m 



(2-2) 



It follows from Picard's theorem that there exists an e > and a neighborhood W of 
the origin in R such that the map 



Thus by the Inverse Function Theorem, (p = er is a coordinate map on some neighbor- 
hood U of m. Let X\, a^denote the coordinate functions of the coordinate system (U, <p). 
Then since 



lemma 2. Any one dimensional distribution is integrable. 

Proof. See proposition 11.1.1 in [3] or proposition 1.53 in [14]. □ 

definition 2. Let (a,b) be an interval on the real line, but a can be —oo and b can 
be +oo. A differ entiable curve 7 : (a, 6) — > M 2 is an integral curve of a vector field X if 
j'(t) = X(j(t)) for all t e (0, 1) where 7' denotes the derivative of 7. 

Thus an integral curve is a curve whose tangent at each point coincides with the vector 
of the vector field at the same point. Since the gradient vector points in the direction of 
steepest descent, perpendicular vectors are produced by the Hamiltonian operator, and 
if we join these Hamiltonian vectors in a head to tail fashion, they line up into integral 
curves. 

DEFINITION 3. An integral curve is called a complete integral curve if its domain is 
E = (-00, +00). 

The following is the authors Theorem 3.3 from pQ. 

theorem 1. Let X be a smooth vector field on the plane, and M the subset of the 
plane on which X is non-zero. Each point of M lies in precisely one maximal integral 
curve, and there is a one parameter family of such integral curves in M. An integral 
curve does not intersect itself. 

Proof. By lemma[2]the distribution m 1— > span(X(m)) is integrable. Proposition 11.2.1 
in [3] asserts that an integrable distribution is involutive. A one dimensional integral 



a(t,a 2 ) =X t (T- 1 (0,a 2 )) 

is well-defined and smooth for (t, 02)6 (—£>£) x W C M 2 . 
Now, a is non-singular at the origin since 





we have 




□ 
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Figure 1. (a) Vector field X on surface patch (b) integral curves of X 
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submanifold of M is a smooth curve in M whose tangent equals the line in the distribution 
at each point of the curve. The Frobenius theorem (theorem 1.60) in |14] asserts that 
there is a cubic coordinate system centred at each point of M, with coordinates xi,X2 
such that x-i = constant are integral submanifolds of M. By theorem 1.64 in |14| . there is 
a unique maximal integral submanifold passing through each point of M. By proposition 
11.3.1 in [3] any integral curve of X is a integral submanifold of M. Consequently each 
point of M lies in precisely one complete integral curve, and there is a one parameter 
family of such curves in M. 
□ 

example 3. Consider the circle (z — l) 2 + x 2 = |. Let l : (x, z) i— > x/z produce a 
one-dimensional image of this circle. At the point x=\,z= \ the tangent to the circle 
passes through the origin, so this is an occluding point of the image. Let t be the image 
coordinate, and calculate the differential 

So the tangent vector to the circle at (5, is 

Id Id 
V ~ ~2dx~ + 2dz 
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Evaluating di{v) we find it is zero. Since a = x, (3 = z at any occluding point, db(v) = 
at any occluding point. 

DEFINITION 4. Let i : (x, y, z) — * (t, s) where t = x/z, s — y/z, and z ^ 0. 

Then 

di( v — + v — + v —) - <— - —)— + (—- V3V ) 9 
dx dy dz a z z 2 dt z z 2 ds 

Thus di(v) \ a = if and only if v is a multiple of a. Thus dt(v) has rank 2 at all points 
on a surface disjoint from the focal plane, except occluding points, where it has rank 1. 

3. Sufficient condition for a discontinuity 

The corollaries to the theorem Q] give us a condition to establish that there is a discon- 
tinuity or occlusion in an image. 

COROLLARY 1. Suppose Mi is a smooth two dimensional manifold in the scene, whose 
tangent planes are everywhere disjoint from the optical centre of a camera. Suppose also 
that the intensity on Mi is smooth. 

(i) Then there is a smooth invertible mapping between vector fields on M\and vector 
fields on the image of M\, 

(ii) Moreover, the non-vanishing vector fields on M\ correspond to the non-vanishing 
vector fields on the image of M\. 

(hi) Each point in a smooth and unoccluded image of a smooth and unoccluded surface 
lies in precisely one maximal integral curve of a smooth image vector field, and there is 
a one parameter family of such integral curves. 

Proof, (i) The differential of t maps a vector field on a smooth surface to a vector field 
on its image. Vector fields on images minus the occluding points can be mapped to vector 
fields on a smooth surface by the inverse differential. 

(ii) That non-vanishing vector fields on Mi correspond to the non-vanishing vector 
fields on the image of Mi, follows because the tangent planes were disjoint from the 
optical centre. 

(hi) From Theorem [TJ each point in a smooth and unoccluded image of a smooth and 
unoccluded surface lies in precisely one maximal integral curve of a smooth image vector 
field, and there is a one parameter family of such integral curves. □ 

The contrapositive is often used in mathematics, mainly for proofs by contradiction, such 
as the irrationality of the square root of two. Consider any predicates P, Q and denote 
notP and notQ by -iP, ~^Q. The contrapositive of P=> Q is ->Q => -P. Since P => Q is 
^PVQ then ^(-P)V^Q which is P V thus -<Q =>• -P. It is the contrapositive that 
enables us to find a discontinuity. 

COROLLARY 2. If an image point lies in more than one maximal integral curve, then 
either the intensity on the surface is not smooth, or the surface is not smooth, or the 
surface is occluded. 

Proof. This is the contrapositive of Theorem HJ or part (iii) of the previous corol- 
lary. □ 
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In other words, we have obtained a sufficient condition for a singularity in intensity, or 
the surface or an occlusion. 

The following lemma gives us a means for finding integral curves in closed form. Its 
relevance to vision was observed by Faugeras (see pgll2 of |6j); those image curves whose 
normals are parallel to the gradient are the level curves of intensity. What follows is an 
alternate statement. 

lemma 3. The Hamiltonian operator H = ^— has integral curves I (x,y) = 

constant 

Proof. Let xo be the point of interest. The level set going through xq is x \ f(x) = f(xo). 
Consider a curve ^(t) in the level set going through x$, so we will assume that 7(0) = x n . 
We have 

/((7(*)) = /(*o)=c 
Now let us differentiate at t — by using the chain rule. We find 

J/(x ) 7 '(0) = 
Equivalently, the Jacobian of / at xq is the gradient at Xq 

V/(x )- 7 '(0) = 

Thus, the gradient off at xq'is perpendicular to the tangent 7'(0) to the curve (and to 
the level set) at that point. Since the curve j(t) is arbitrary, it follows that the gradient 
is perpendicular to the level set. 

Thus the integral curves of the Hamiltonian are level curves of intensity, that is curves 
with / (x, y) = constant. □ 

Thus the Hamiltonian is an operator whose integral curves we can calculate in closed 
form, circumventing all the errors associated with numerical differentiation, numerical 
equation solving, and even floating point arithmetic. However there is an exception at a 
finite number of points where the vector field on a surface vanishes. 

An image is only a discrete sample of measurements of light reflected from a surface. 
If a pixel has a particular measurement value, none of its neighbors necessarily has the 
same value. It is therefore not feasible to calculate level curves by trying to chain together 
pixels that have the same measurement value. A more sensible procedure is to calculate 
the set of image points whose measurement value is less than the particular measurement 
value, and then compute the boundary of this set (Figure [3j. 

definition 5. Let 

i c = d{p\i( P )<c} 

where d denotes the boundary of the set. 

theorem 2 . If p 6 I c H Id with c ^ c' then either the intensity on the surface is not 
smooth, or the surface is not smooth, or the surface is occluded. 

Proof. I c and I c < consist of level curves of intensity, which are maximal integral curves 
of the Hamiltonian operator (from Lemma [3]). Since p lies in these two distinct maximal 
integral curves, it follows from Corollary [2] that either the intensity on the surface is not 
smooth, or the surface is not smooth, or the surface is occluded. □ 
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Figure 3. Level curves of smooth I are boundaries of J(p) < c 
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The surfaces of objects in scenes have another property, namely they are bounded, and 
can be considered to be closed sets. A closed and bounded subset of M. m is called compact. 
A more general definition of compactness can be given in the more general setting of 
topological spaces. A collection of open sets is said to cover a set S if S is contained 
in the union of all the open sets in the collection. A set S is called compact if every 
collection of open sets covering S has a finite sub-collection that covers S. This is not an 
easy property to grasp, but it is a very important one. Theorem 6.3 in [13] asserts that 
the closed and bounded subsets of M. m coincide with the compact subsets. 

Surfaces can therefore be considered to be compact manifolds, but this includes parts 
of the surface that are not visible, or in contact with other surfaces like the ground. 

It can be shown that all maximal integral curves of a compact smooth manifold are 
complete |14| . Thus a hypothesis that an image is a smooth image of an unoccluded, 
smooth compact surface implies that all maximal integral curves are closed curves. We 
intuitively know that this follows from an absurd assumption, namely that all of a surface 
is visible. However, since we want to reject such a hypothesis, we must employ the 
consequences which are most certain to lead to a contradiction. This is why we will 
generate only closed curves, by completion of curves if necessary. The phenomenon of 
completion occurs in human vision, and can be demonstrated vividly by an optical illusion 
called the Kanisza triangle. 
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An algorithm was implemented based on Theorem [2j using values c in constant in- 
crements. This algorithm used no floating point arithmetic. This was run on a super- 
computer belonging to the University of Melbourne. The results of this were surprising. 
If very small increments are chosen then almost the entire image is returned. What this 
shows is that the algorithm can detect discontinuities that are dense, not just discon- 
tinuities that fall on a curve. However if larger increments are chosen, then edges are 
returned. The discontinuities found do not surround objects completely. 

4. Relation to threshold's 

We will now show that the existence of a neighboring point and a threshold such that 
the difference in measurements exceeds the threshold is a necessary but not sufficient 
condition for an occlusion or singularity to exist. 

LEMMA 4. If p G I c H I c ' then there is a neighboring point q such that 

I(p)-I(q) > \c-c'\ 

Proof. If p E I c n 7 c 'then p G d {r | J(r) < c}and p G d {r \ I (r) < c'} 
Thus I (p) > c and I (p c ) < c for some neighboring point p c . 
Also / (p) > c'and / (p C ' ) < c'for some neighboring point p c > . 
Thus I (p) - I (p c > ) > c- d and I (p) - I (p c ) > d - c. 

It follows that p has a neighboring point q such that I (p) — I (q) > | c — d | □ 

We have shown that if the sufficient condition for an occlusion or discontinuity in an 
image or scene is met at a point in an image (see Theorem 3.3 in PhD thesis), then 
there is a neighboring point and a threshold that must be exceeded. In other words we 
have obtained a necessary condition for a sufficient condition. Unfortunately this does 
not suffice because the condition is not necessary and sufficient. 

example 4. I (p) — I (q) >| c — d |does not imply p E I c H I c > Let I (p) — 5 I (p c ) = 3 
J (p c >) = 3c = 2c' = l Thus I (p) — I (p c >) > c — d and I (p) — I (p c ) > d — c However 
I (Pc) > c so p is not in I c 

What this lemma and example tells us is that threshold's will give us a set of occlu- 
sions and discontinuities, but will also include points that are neither occlusions nor 
discontinuities. We now give a stronger condition that is more useful. 

lemma 5. Lemma p G I c D I C ' with c ^ d if and only if at point p there exist two 
neighboring points p' andp" and two thresholds e > and e' > such that I (p) — I (p 1 ) > 
e and I (p) — I (p") > e' and e + I (p 7 ) ± e' + 1 (p") 

Proof. Suppose at point p there exist two neighboring points p' and p"and thresholds 
c, e' such that I (p) - I (p') > e > and I(p) - I (p") > e' > 0. 
Let c = e + I (p') 
Then I (p) > e + I (p') = c 
And c > I (p 1 ) since e > 

Thus I (p) > c and I (p') < c. It follows that p E I c = d {p \ I (p) < c}. 
Letting d = e' + I{p") it follows similarly that p G I c > 

From c = e+I (p') ^ e'+I (p") = d it follows c ^ d . Hence p G I c nJ c / with c ^ d . □ 
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Suppose peJ c fl I c > with d . Then 

I(p)>c>I (p c ) ^I(p)-I (p c ) > c - / Oc) > 

and 

I(p)>c'>I (pc) ^I(p)-I (pc) >c'-I (pc) > 

Letting e = c — I (p c ) and e' = c' — I (pc) and p' = p c and p" — pc it follows that 
I(p)-I (p r ) > e > and / (p) - / (p") > e' > 0. 

The reader should note that the neighboring points p' and p" can coincide, providing 

Consider H = (— J^, J|) which is a Hamiltonian operator. The integral curves of H 
are level curves / (x, t). 

The lemma shows that if (x,t) has two neighboring points (xi,ti)i — 1,2 such that 
/ (x, t) — I (xi,U) > Si > with Ei + 1 (x\, ti) 7^ £2 + 1 (%2, £2) then there is an occlusion 
or discontinuity &t(x,t). 

So if we used two consecutive video frames and two different thresholds, we could apply 
the test to the time increments to determine a set of occlusions and discontinuities. We 
could also use three consecutive video frames and two different thresholds, and apply the 
test. 

Note that we can apply any smooth differential operator to the image function. For 
example we could differentiate the green component and set I (x,y) = (§7) (x,y). Then 
applying the same algorithm to I would determine a set of occlusions and discontinuities. 
We can also use higher order smooth differential operators, or finite difference approx- 
imations to smooth differential operators. In fact we can use absolutely any process 
that transforms an arbitrary smooth function into another arbitrary smooth function, to 
substitute in a new I into the algorithm. 

5. applications to image analysis or computer vision 

Two main components of a camera or the eye are a lens and a collection of light 
sensors on a surface, called the retina. The retina of a camera is usually planar, and will 
be referred to as the retinal plane. The camera forms an image of the scene in front of 
it. The correspondence between a point in the scene and a point in the retinal plane can 
be approximately given by a straight line through the optical centre of the camera (see 
figure [U ) • 

This model can be justified by geometric optics. A convex lens with two spherical 
surfaces whose thickness can be neglected is called a thin lens in geometric optics. The 
lens of a camera is usually a series of thin lenses (called a thick lens in geometric optics, 
see page 25 of [15|). The paraxial or Gaussian assumption of geometric optics is that the 
angle between a scene point and the optical axis of the lens system is small. It is known 
in geometric optics that, under the paraxial assumption, the vertical distance between 
the scene point and the optical axis and the vertical distance between its focussed image 
point and the optical axis is a constant, called the magnification of the lens system (see 
page 28 of p~5j). 

The elementary physical concept of reflection is that a light ray incident to a surface 
will be reflected at an angle to the surface normal equal to the angle of incidence, such 
that incident and exitant rays lie in the same plane through the normal vector. A mirror 
is an example of such an ideal, or specular reflector. The microscopic structure of most 
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Figure 4. 
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materials in scenes is not smooth, so a scene surface will scatter light in different di- 
rections. Most materials are also not perfectly homogeneous on a microscopic scale, and 
thus scatter light rays that penetrate the surface by refraction and reflection at bound- 
aries between regions of different refractive indices. Scattered rays may reemerge near 
the point of entry, and so contribute to diffuse reflection. Snow and layers of white paint 
are examples of materials with this behaviour |9j. 

A surface is called Lambertian if it appears equally bright from all directions, regard- 
less of how it is irradiated, and reflects all incident light [9j. This does not imply that 
different points on the same surface have the same intensity, because the value of the 
intensity depends on other variables such as the angle between the surface normal and 
a light source. When a smooth lambertian surface is rendered by computer graphics, its 
appearance is like a dull smooth plastic [12j. 

Most materials in scenes are neither specular reflectors nor Lambertian reflectors, but 
a combination of both. A surface rendered by computer graphics will appear shinier as 
the specular component is increased [12, 7\. The appearance of texture (to the eye) is 
due to particularly rapid variations in normal vectors over small regions. For example, 
a random pattern of normal vectors when rendered by computer graphics can produce 
an effect of fog. Various repetitive patterns of normal vectors can be used to render real 
materials, such as bricks by computer graphics [7j- The Lambertian assumption can also 
be used to directly recover surface shape from image shading [4j [5] . 

Owing to the microstructure of a scene surface, the light radiated from the surface 
undergoes extremely rapid intensity changes over small regions. This can be confirmed 
by examining gray values of neighbouring points in a real image. 

Let i : (x,y,z) — > (t, s) where t = f , s = |, and z ^ 0. For a camera with retina at 
z = 1 and optical centre at the origin, the function i represents image formation. The 
point (x, y, z) is being viewed, and its image is formed at (f , |). 

Note that the co-ordinates (t, s) must be (0, 0) at the point where the optical axis 
meets the retinal plane perpendicularly. It is preferred to find this point in the image 
by some procedure. Zooming the camera and finding the point around which the image 
expands or contracts may be one way. The unit of distance is approximately the distance 
between the pupil and the ccd. 

For the purpose of stereo or binocular vision, points in an image where the surface 
being viewed is not smooth need to be distinguished from occluding points. This is 
because under the assumption of Lambertian shading, different viewpoints of the same 
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point have similar intensity values. Points where the surface being viewed is not smooth 
possess invariance to viewpoint. On the other hand occluding points move on the surface 
when they are viewed from a different viewpoint. Occluding points can be eliminated by 
filtering out zeroes of the Hamiltonian. This does require floating point arithmetic. For 
other purposes such as robot navigation where we are just looking for empty spaces this 
is not an issue. 

Also note that we could use any discrete topology on the plane to define the neighbors 
of a point. For example we could use 4 neighbors north, west, east and south of each 
pixel. Or we could use 8 neighbors. Or some other topology would also do. 
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